Create a connection between a Splunk forwarder and the Forwarders service

To get data from a Splunk heavy forwarder or universal forwarder into a data pipeline in the , you must first create an SSL-secured connection between the forwarder and the Forwarders service. You'll need to generate a self-signed client certificate or use an existing SSL certificate, upload it to the DSP Forwarders service, and configure your forwarder to use that certificate. You can then use the Forwarders Service source function to get data from the forwarder into a DSP pipeline.

Generate a client certificate for the Forwarders service

Generate and upload a self-signed client certificate. This certificate will be used to secure the connection between your forwarder and DSP's forwarder service.

Alternatively, you can use an existing SSL certificate. If you want to use an existing certificate, skip to Step 5 to upload the certificate to the forwarder service.

Prerequisites

• Choose a name to identify your Splunk instance. Use this name for the generated key, certificate signing request (CSR), and PEM files.
• Choose an email address to associate to the certificate. The email address is included in the name of the certificate.
• If you are using an existing SSL certificate, then that SSL certificate must be in PEM format. You cannot upload more than one certificate at a time, so make sure that the file you are uploading only contains one certificate.

Steps

Follow these steps to secure the connection between DSP and your Splunk forwarders. If you are using an existing SSL certificate, skip to step 5.

1. From the command line, generate a private key to sign your certificates.

   openssl genrsa -out my_forwarder.key 2048

2. If you do not have a .rnd file in your home directory, you must use openSSL to generate a new file.

   openssl rand -writerand ~/.rnd

3. Use your private key to generate a CSR file.

   openssl req -new -key "my_forwarder.key" -out "my_forwarder.csr" -subj "/C=US/ST=CA/O=my_organization/CN=my_forwarder/emailAddress=email@example.com"

4. Sign your CSR file with your newly created private key and create a client certificate that expires in 2 years.

   openssl x509 -req -days 730 -in "my_forwarder.csr" -signkey "my_forwarder.key" -out "my_forwarder.pem" -sha256

   Your client certificate cannot be valid longer than 5 years.

5. Use the Splunk Cloud Services CLI tool to format your certificate and upload your certificate to your tenant. For more information about the Splunk Cloud Services CLI, see Get started with the Splunk Cloud Services CLI.

   scloud forwarders add-certificate --input-datafile my_forwarder.pem

After the key is added to the tenant, the server response shows details about the certificate including the hash, subject, issuer, and validity dates.

By default, you can add up to 10 certificates to your tenant.

After setting up the client certificate in the Forwarders service, you'll need to configure your Splunk forwarder to use that certificate.

Configure your forwarder to use the client certificate

1. On the host that forwards the data that you want to collect, open a shell or command prompt or PowerShell window.
2. Concatenate your private and public keys into one file.

   cat my_forwarder.pem my_forwarder.key > my_forwarder-keys.pem
   

3. Navigate to the configuration directory for the forwarder.

   cd $SPLUNK_HOME/etc/system/local

4. Open outputs.conf in etc/system/local for editing and add the following stanzas. If outputs.conf does not exist, create it. See outputs.conf.spec in the Admin Manual for more information about these settings.

   [tcpout]
   defaultGroup = my_defaultGroup
   
   [tcpout:my_defaultGroup]
   server=<ip-address-of-controller-node>:9997
   
   clientCert=/path/to/my_forwarder-keys.pem
   sslVerifyServerCert=false 
   

5. (Optional) If you want to use indexer acknowledgment, set useACK to true in outputs.conf.

   useACK=true
   

If you are sending data from a large number of forwarders and you are experiencing throughput issues, you should increase the number of replicas for the ingest-s2s deployment in your Kubernetes cluster. You can increase the number of replicas by adding more nodes to your cluster. See Install the Data Stream Processor for instructions on how to join a new node to your cluster.

The forwarder starts forwarding data to DSP. You can now use the forwarder as a data source by creating a pipeline that starts with the Forwarders Service source function.

• For instructions on how to build a data pipeline, see the Building a pipeline chapter in the Use the Data Stream Processor manual.
• For information about the Forwarders Service source function, see Get data from Forwarders Service in the Function Reference manual.
• For instructions on how to process data from a universal forwarder so that the events are not truncated or grouped in unexpected ways, see Process data from a universal forwarder in DSP.